Khadas VIM3

Posted on 2019-12-31 | Edited on 2020-01-03 | In AI , EdgeComputing

Great… Today is the LAST day of 2019. Happy new year everyone. How time flies!!! Let’s welcome a fresh and fruitiful 2020. Due to my ENDLESS jet lag and this fabulous edge computing single board computer Khadas VIM3, I’m writing this blog in such early morning.

1. About Khadas VIM3

Khadas VIM3 is a super computer based on Amlogic A311D. Before we start, let’s carry out two simple comparisons.

1.1 Raspberry Pi 4 Model B vs. Khadas VIM3 vs. Jetson Nano Developer Kit

Please refer to hackerboards comparisons.

1.2 Amlogic S912 vs Amlogic A311D

Please refer to cnx-software embedded system news.

2. Ubuntu XFCE for Khadas VIM3

Let’s try out Ubuntu XFCE on Khadas VIM3.

2.1 Ubuntu XFCE Installation

As of today, I just downloaded VIM3_Ubuntu-xfce-bionic_Linux-5.5-rc2_arm64_SD-USB_V0.8.2-20200103.img, and flash it onto a TF card as:

➜  khadas sudo dd bs=4M if=VIM3_Ubuntu-xfce-bionic_Linux-5.5-rc2_arm64_SD-USB_V0.8.2-20200103.img of=/dev/mmcblk0 conv=fsync
1093+0 records in
1093+0 records out
4584374272 bytes (4.6 GB, 4.3 GiB) copied, 206.586 s, 22.2 MB/s

2.2 SSH Into Khadas VIM3

➜  ~ ssh khadas@192.168.1.95
The authenticity of host '192.168.1.95 (192.168.1.95)' can't be established.
ECDSA key fingerprint is SHA256:Q59XrIX7bSWsphZCpgHBSnVH5ETgCY9iLfDEuvRKtOw.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.95' (ECDSA) to the list of known hosts.
khadas@192.168.1.95's password: 

Welcome to Fenix 0.8.2 Ubuntu 18.04.3 LTS Linux 5.5.0-rc2  
 _  ___               _            __     _____ __  __ _____ 
| |/ / |__   __ _  __| | __ _ ___  \ \   / /_ _|  \/  |___ / 
| ' /| '_ \ / _` |/ _` |/ _` / __|  \ \ / / | || |\/| | |_ \ 
| . \| | | | (_| | (_| | (_| \__ \   \ V /  | || |  | |___) |
|_|\_\_| |_|\__,_|\__,_|\__,_|___/    \_/  |___|_|  |_|____/ 
                                                             

 * Website:        https://www.khadas.com
 * Documentation:  https://docs.khadas.com
 * Forum:          https://forum.khadas.com
khadas@Khadas:~$ uname -a
Linux Khadas 5.5.0-rc2 #0.8.2 SMP PREEMPT Fri Jan 3 09:48:06 CST 2020 aarch64 aarch64 aarch64 GNU/Linux
khadas@Khadas:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.3 LTS
Release:	18.04
Codename:	bionic

......
Installing Amlogic flash-tool...

===============================================

Host PC: Ubuntu 18.04

===============================================

Installing USB rules...
Installing flash-tool...
Done!

Installing Rockchip flash-tool...

===============================================

Host PC: Ubuntu 18.04

===============================================

Installing USB rules...
Installing flash-tool...
Installing configuration...
Done!

Installing ToneBoard burn-tool...

===============================================

Host PC: Ubuntu 18.04

===============================================

Installing USB rules...
Installing toneboard-burn-tool...
Done!
Installing Khadas burn-tool...
Done!

3. OpenVINO Installation

Let’s just follow this Intel official documentation Arm* 64 Single Board Computers and the Intel® Neural Compute Stick 2 (Intel® NCS 2).

OpenVINO on Raspberry Pi 4 with Movidius Neural Compute Stick II

Posted on 2019-12-28 | Edited on 2020-01-09 | In AI , EdgeComputing

Merry Christmas and happy new year everybody. I’ve been back to Vancouver for several days. These days, I’m updating this blog FIRSTLY written in September 2019. 2020 is coming, and we’re getting 1 year older. A kind of sad hum?

Okay… No matter what, let’s enjoy the song first: WE ARE YOUNG. Today, I joined Free Software Foundation and start my journey of supporting Open Source Software BY CASH. For me, it’s not about poverity or richness. It’s ALL about FAITH.

To write something about Raspberry Pi is to say GOOD BYE to my Raspberry Pi 3B, and WELCOME Raspberry Pi 4 at the same time. Our target today is to build an AI edge computing end as the following Youtube video:

1. About Raspberry Pi 4

1.1 Raspberry Pi 4 vs. Raspberry Pi 3B+

Before we start, let’s carry out a simple comparison between Raspberry Pi 4 and Raspberry Pi 3B+.

1.2 Raspbian Installation

➜  raspbian sudo dd bs=4M if=2019-09-26-raspbian-buster.img of=/dev/mmcblk0 conv=fsync 
913+0 records in
913+0 records out
3829399552 bytes (3.8 GB, 3.6 GiB) copied, 144.192 s, 26.6 MB/s

1.2 BCM2711 is detected as BCM2835

pi@raspberrypi:~ $ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 1
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 2
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

processor	: 3
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3

Hardware	: BCM2835
Revision	: c03111
Serial		: 100000006c0c9b01
Model		: Raspberry Pi 4 Model B Rev 1.1

This issue seems to be a well-known bug. Raspberry Pi 4’s specification can be retrieved from The MagPi Magazine. More details about the development history of Raspberry Pi can be found on Wikipedia.

2. Movidius Neural Compute Stick on Raspberry Pi 4

Then, we just follow the following 2 blogs Run NCS Applications on Raspberry Pi and Adding AI to the Raspberry Pi with the Movidius Neural Compute Stick to test out Intel Movidius Neural Compute Stick 2:

Intel Movidius Neural Compute Stick 2	Intel Movidius Neural Compute Stick 1

Intel Movidius Neural Compute Stick 1 is NOT listed on Intel’s official website any more. But github support for Intel Movidius Neural Compute Stick 1 can be found at https://github.com/movidius/ncsdk.

2.1 NCSDK Installation

We FIRST need to have ncsdk installed. Yup, here, as described in Run NCS Applications on Raspberry Pi, we carry out the installation directly under folder ...../ncsdk/api/src.

1
2
3

➜  src git:(ncsdk2) ✗ make -j4
NCSDK FW successfully installed
➜  sudo make install

2.2 Test NCSDK Example Apps

➜  python hello_ncs.py
Hello NCS! Device opened normally.
Goodbye NCS! Device closed normally.
NCS device working.

2.3 mvnc Python Package

pi@raspberrypi:~ $ python
Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mvnc
>>> mvnc.__file__
'/usr/local/lib/python3.7/dist-packages/mvnc/__init__.py'
>>>

3. Transitioning from Intel Movidius Neural Compute SDK to Intel OpenVINO

By following Intel’s official documentation Transitioning from Intel® Movidius™ Neural Compute SDK to Intel® Distribution of OpenVINO™ toolkit, we are transitioning to OpenVINO, which supports both Intel NCS 2 and the original NCS.

3.1 Install OpenVINO for Raspbian

For the installation details of OpenVINO, please refer to the following 2 documentations:

We now extract the MOST up-to-date l_openvino_toolkit_runtime_raspbian_p_2019.3.334.tgz under folder /opt/intel/openvino. Let’s take a brief look at this folder:

pi@raspberrypi:/opt/intel/openvino $ ls
bin  deployment_tools  documentation  inference_engine  install_dependencies  licensing  opencv  python
pi@raspberrypi:/opt/intel/openvino $ ls deployment_tools/
inference_engine

Clearly, by comparing with OpenVINO™ Toolkit - Deep Learning Deployment Toolkit repository, we know that the open source version of deployment_tools contains some more content than the trimmed version for Raspbian. We’ll use model-optimizer for sure. Therefore, we checked out dldt, and put two subfolders model-optimizer and tools under folder /opt/intel/openvino/deployment_tools.

1 2	pi@raspberrypi:/opt/intel/openvino/deployment_tools $ ls inference_engine open_model_zoo tools

3.2 Build OpenVINO Samples

Before start building OpenVINO samples, please have OpenCV built from source and installed.

Note: Be sure to enable -DCMAKE_CXX_FLAGS=’-march=armv7-a’ while building dldt/samples.

Those samples are ALL for C++. For Python samples, you may have to figure out where to put openvino and where should those .so files put under. Anyway, the final step is to test as follows:

pi@raspberrypi:/opt/intel/openvino/inference_engine/samples/python_samples $ python hello_query_device/hello_query_device.py 
Available devices:
	Device: MYRIAD
	Metrics:
		AVAILABLE_DEVICES: 1.4.3-ma2480
		FULL_DEVICE_NAME: Intel Movidius Myriad X VPU
		SUPPORTED_METRICS: AVAILABLE_DEVICES, FULL_DEVICE_NAME, SUPPORTED_METRICS, SUPPORTED_CONFIG_KEYS, OPTIMIZATION_CAPABILITIES, RANGE_FOR_ASYNC_INFER_REQUESTS
		SUPPORTED_CONFIG_KEYS: VPU_HW_STAGES_OPTIMIZATION, VPU_LOG_LEVEL, VPU_PRINT_RECEIVE_TENSOR_TIME, VPU_NETWORK_CONFIG, VPU_COMPUTE_LAYOUT, VPU_CUSTOM_LAYERS, VPU_IGNORE_IR_STATISTIC, VPU_MYRIAD_FORCE_RESET, VPU_MYRIAD_PLATFORM, EXCLUSIVE_ASYNC_REQUESTS, LOG_LEVEL, PERF_COUNT, CONFIG_FILE, DEVICE_ID
		OPTIMIZATION_CAPABILITIES: FP16
		RANGE_FOR_ASYNC_INFER_REQUESTS: 3, 6, 1

	Default values for device configuration keys:
		VPU_HW_STAGES_OPTIMIZATION: UNSUPPORTED TYPE
		VPU_LOG_LEVEL: UNSUPPORTED TYPE
		VPU_PRINT_RECEIVE_TENSOR_TIME: UNSUPPORTED TYPE
		VPU_NETWORK_CONFIG: UNSUPPORTED TYPE
		VPU_COMPUTE_LAYOUT: UNSUPPORTED TYPE
		VPU_CUSTOM_LAYERS: UNSUPPORTED TYPE
		VPU_IGNORE_IR_STATISTIC: UNSUPPORTED TYPE
		VPU_MYRIAD_FORCE_RESET: UNSUPPORTED TYPE
		VPU_MYRIAD_PLATFORM: UNSUPPORTED TYPE
		EXCLUSIVE_ASYNC_REQUESTS: UNSUPPORTED TYPE
		LOG_LEVEL: UNSUPPORTED TYPE
		PERF_COUNT: UNSUPPORTED TYPE
		CONFIG_FILE: UNSUPPORTED TYPE
		DEVICE_ID: UNSUPPORTED TYPE

3.3 OpenVINO Test

3.3.1 Object Detection

We then download two model files as given in the blog Install OpenVINO™ toolkit for Raspbian* OS.

Afterwards, start running object_detection_sample_ssd as follows:

pi@raspberrypi:/opt/intel/openvino/inference_engine/samples/build $ ./armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg 
[ INFO ] InferenceEngine: 
	API version ............ 2.1
	Build .................. custom_releases/2019/R3_cb6cad9663aea3d282e0e8b3e0bf359df665d5d0
	Description ....... API
Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     me.jpg
[ INFO ] Loading Inference Engine
[ INFO ] Device info: 
	MYRIAD
	myriadPlugin version ......... 2.1
	Build ........... 30677
[ INFO ] Loading network files:
	face-detection-adas-0001.xml
	face-detection-adas-0001.bin
[ INFO ] Preparing input blobs
[ INFO ] Batch size is 1
[ INFO ] Preparing output blobs
[ INFO ] Loading model to the device
[ INFO ] Create infer request
[ WARNING ] Image is resized from (924, 1280) to (672, 384)
[ INFO ] Batch size is 1
[ INFO ] Start inference
[ INFO ] Processing output blobs
[0,1] element, prob = 0.870117    (421,202)-(745,608) batch id : 0 WILL BE PRINTED!
[1,1] element, prob = 0.0268555    (-55,556)-(234,1100) batch id : 0
[2,1] element, prob = 0.0219727    (20,925)-(239,1340) batch id : 0
[3,1] element, prob = 0.0200195    (-18,-144)-(179,581) batch id : 0
[4,1] element, prob = 0.0180664    (636,315)-(757,471) batch id : 0
[5,1] element, prob = 0.0170898    (-27,672)-(137,1217) batch id : 0
[6,1] element, prob = 0.0170898    (74,976)-(364,1322) batch id : 0
[7,1] element, prob = 0.0161133    (291,958)-(349,1122) batch id : 0
[8,1] element, prob = 0.0161133    (329,963)-(391,1126) batch id : 0
[9,1] element, prob = 0.0161133    (203,928)-(423,1346) batch id : 0
[10,1] element, prob = 0.015625    (694,1034)-(732,1140) batch id : 0
[11,1] element, prob = 0.015625    (832,1121)-(937,1295) batch id : 0
[12,1] element, prob = 0.015625    (219,39)-(388,627) batch id : 0
[13,1] element, prob = 0.015625    (76,785)-(357,1160) batch id : 0
[14,1] element, prob = 0.0146484    (355,616)-(475,778) batch id : 0
[15,1] element, prob = 0.0146484    (240,779)-(331,1048) batch id : 0
[16,1] element, prob = 0.0136719    (0,893)-(24,984) batch id : 0
[17,1] element, prob = 0.0136719    (658,306)-(752,405) batch id : 0
[18,1] element, prob = 0.0136719    (204,1198)-(267,1275) batch id : 0
[19,1] element, prob = 0.0136719    (322,611)-(430,794) batch id : 0
[20,1] element, prob = 0.0136719    (171,244)-(423,798) batch id : 0
[21,1] element, prob = 0.0136719    (252,798)-(539,1177) batch id : 0
[22,1] element, prob = 0.0136719    (133,875)-(301,1420) batch id : 0
[23,1] element, prob = 0.0136719    (443,202)-(802,1022) batch id : 0
[24,1] element, prob = 0.0126953    (718,466)-(757,540) batch id : 0
[25,1] element, prob = 0.0126953    (166,527)-(206,594) batch id : 0
[26,1] element, prob = 0.0126953    (694,511)-(731,590) batch id : 0
[27,1] element, prob = 0.0126953    (705,441)-(771,555) batch id : 0
[28,1] element, prob = 0.0126953    (272,962)-(322,1117) batch id : 0
[29,1] element, prob = 0.0126953    (268,486)-(376,688) batch id : 0
[30,1] element, prob = 0.0126953    (323,489)-(422,690) batch id : 0
[31,1] element, prob = 0.0126953    (277,607)-(384,796) batch id : 0
[32,1] element, prob = 0.0126953    (233,709)-(342,924) batch id : 0
[33,1] element, prob = 0.0126953    (256,672)-(406,957) batch id : 0
[34,1] element, prob = 0.0126953    (252,1043)-(431,1214) batch id : 0
[35,1] element, prob = 0.0117188    (695,466)-(733,539) batch id : 0
[36,1] element, prob = 0.0117188    (673,514)-(709,587) batch id : 0
[37,1] element, prob = 0.0117188    (370,1005)-(399,1078) batch id : 0
[38,1] element, prob = 0.0117188    (670,256)-(753,346) batch id : 0
[39,1] element, prob = 0.0117188    (151,500)-(221,610) batch id : 0
[40,1] element, prob = 0.0117188    (447,489)-(511,608) batch id : 0
[41,1] element, prob = 0.0117188    (126,625)-(200,719) batch id : 0
[42,1] element, prob = 0.0117188    (299,626)-(387,709) batch id : 0
[43,1] element, prob = 0.0117188    (164,673)-(251,747) batch id : 0
[44,1] element, prob = 0.0117188    (398,850)-(453,1008) batch id : 0
[45,1] element, prob = 0.0117188    (425,851)-(476,1008) batch id : 0
[46,1] element, prob = 0.0117188    (310,915)-(371,1077) batch id : 0
[47,1] element, prob = 0.0117188    (357,925)-(413,1073) batch id : 0
[48,1] element, prob = 0.0117188    (378,923)-(435,1074) batch id : 0
[49,1] element, prob = 0.0117188    (468,906)-(521,1070) batch id : 0
[50,1] element, prob = 0.0117188    (284,1016)-(357,1188) batch id : 0
[51,1] element, prob = 0.0117188    (203,-95)-(283,182) batch id : 0
[52,1] element, prob = 0.0117188    (269,378)-(374,595) batch id : 0
[53,1] element, prob = 0.0117188    (-22,575)-(71,831) batch id : 0
[54,1] element, prob = 0.0117188    (318,698)-(433,911) batch id : 0
[55,1] element, prob = 0.0117188    (221,716)-(395,1258) batch id : 0
[56,1] element, prob = 0.0117188    (859,771)-(987,1518) batch id : 0
[57,1] element, prob = 0.0107422    (145,466)-(185,527) batch id : 0
[58,1] element, prob = 0.0107422    (166,463)-(204,521) batch id : 0
[59,1] element, prob = 0.0107422    (185,477)-(224,537) batch id : 0
[60,1] element, prob = 0.0107422    (214,477)-(250,540) batch id : 0
[61,1] element, prob = 0.0107422    (458,478)-(488,536) batch id : 0
[62,1] element, prob = 0.0107422    (151,537)-(177,594) batch id : 0
[63,1] element, prob = 0.0107422    (460,522)-(488,590) batch id : 0
[64,1] element, prob = 0.0107422    (481,523)-(509,588) batch id : 0
[65,1] element, prob = 0.0107422    (716,512)-(755,592) batch id : 0
[66,1] element, prob = 0.0107422    (129,581)-(158,641) batch id : 0
[67,1] element, prob = 0.0107422    (150,579)-(179,635) batch id : 0
[68,1] element, prob = 0.0107422    (171,583)-(201,641) batch id : 0
[69,1] element, prob = 0.0107422    (457,564)-(491,658) batch id : 0
[70,1] element, prob = 0.0107422    (481,581)-(510,647) batch id : 0
[71,1] element, prob = 0.0107422    (652,565)-(689,650) batch id : 0
[72,1] element, prob = 0.0107422    (674,571)-(710,650) batch id : 0
[73,1] element, prob = 0.0107422    (694,571)-(734,649) batch id : 0
[74,1] element, prob = 0.0107422    (143,640)-(188,705) batch id : 0
[75,1] element, prob = 0.0107422    (191,743)-(224,805) batch id : 0
[76,1] element, prob = 0.0107422    (437,843)-(465,920) batch id : 0
[77,1] element, prob = 0.0107422    (-1,943)-(22,1037) batch id : 0
[78,1] element, prob = 0.0107422    (414,1007)-(443,1080) batch id : 0
[79,1] element, prob = 0.0107422    (896,1206)-(924,1279) batch id : 0
[80,1] element, prob = 0.0107422    (678,200)-(743,294) batch id : 0
[81,1] element, prob = 0.0107422    (70,374)-(190,512) batch id : 0
[82,1] element, prob = 0.0107422    (422,429)-(485,563) batch id : 0
[83,1] element, prob = 0.0107422    (106,477)-(213,629) batch id : 0
[84,1] element, prob = 0.0107422    (428,477)-(479,615) batch id : 0
[85,1] element, prob = 0.0107422    (648,498)-(727,605) batch id : 0
[86,1] element, prob = 0.0107422    (703,490)-(769,614) batch id : 0
[87,1] element, prob = 0.0107422    (16,531)-(150,712) batch id : 0
[88,1] element, prob = 0.0107422    (87,546)-(150,685) batch id : 0
[89,1] element, prob = 0.0107422    (127,552)-(197,653) batch id : 0
[90,1] element, prob = 0.0107422    (425,557)-(481,671) batch id : 0
[91,1] element, prob = 0.0107422    (437,509)-(516,700) batch id : 0
[92,1] element, prob = 0.0107422    (467,549)-(524,678) batch id : 0
[93,1] element, prob = 0.0107422    (633,551)-(704,664) batch id : 0
[94,1] element, prob = 0.0107422    (416,611)-(487,735) batch id : 0
[95,1] element, prob = 0.0107422    (466,602)-(530,734) batch id : 0
[96,1] element, prob = 0.0107422    (124,673)-(198,759) batch id : 0
[97,1] element, prob = 0.0107422    (156,705)-(260,849) batch id : 0
[98,1] element, prob = 0.0107422    (54,828)-(137,933) batch id : 0
[99,1] element, prob = 0.0107422    (451,812)-(498,960) batch id : 0
[100,1] element, prob = 0.0107422    (470,843)-(520,1004) batch id : 0
[101,1] element, prob = 0.0107422    (447,897)-(500,1066) batch id : 0
[102,1] element, prob = 0.0107422    (641,966)-(706,1122) batch id : 0
[103,1] element, prob = 0.0107422    (164,85)-(233,410) batch id : 0
[104,1] element, prob = 0.0107422    (188,497)-(307,688) batch id : 0
[105,1] element, prob = 0.0107422    (205,445)-(357,715) batch id : 0
[106,1] element, prob = 0.0107422    (230,602)-(341,791) batch id : 0
[107,1] element, prob = 0.0107422    (274,785)-(390,1021) batch id : 0
[108,1] element, prob = 0.0107422    (26,709)-(209,1239) batch id : 0
[109,1] element, prob = 0.0107422    (309,868)-(485,1410) batch id : 0
[110,1] element, prob = 0.0107422    (229,981)-(575,1306) batch id : 0
[111,1] element, prob = 0.0107422    (29,141)-(518,883) batch id : 0
[ INFO ] Image out_0.bmp created!
[ INFO ] Execution successful

[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool

me.jpg	out_0.bmp

3.3.2 Image Classification

In this section, we are going to test out another OpenVINO example: Image Classification C++ Sample Async. After reading this blog, it wouldn’t be hard for us to notice the MOST important thing we’re missing here is the model file alexnet_fp32.xml. Let’s just keep it in mind for now.

And, let’s review a bit about our previous example: Object Detection – we downloaded face-detection-adas-0001 model from online and use it directly. So, questions:

Are we able to download alexnet_fp32.xml from online this time again?
Where can we download a whole bunch of open source models?

3.3.2.1 Open Model Zoo

It wouldn’t be hard for us to google out OpenVINO™ Toolkit - Open Model Zoo repository, under which model face-detection-adas-0001 is just sitting there. However, face-detection-adas-0001.xml and face-detection-adas-0001.bin are missing.

1 2	$ ls face-detection-adas-0001/ description face-detection-adas-0001.prototxt model.yml

Let’s checkout open_model_zoo and put it under folder /opt/intel/openvino/deployment_tools.

1 2	pi@raspberrypi:/opt/intel/openvino/deployment_tools $ ls inference_engine model_optimizer open_model_zoo tools

It’s seems that each folder under intel and public contains the detailed info of each model. For instance, file intel/face-detection-adas-0001/model.yml contains all the info about model face-detection-adas-0001.

However, what if ONLY caffe model is provided? In order to use those models on movidius, we need to generate two files, one .xml and one .bin, which are optimized for movidius, from some paticular caffe models. Intel OpenVINO toolkit issue 798441 provided a way to generate these two files.

3.3.2.2 Download Caffe Model Files

pi@raspberrypi:/opt/intel/openvino/deployment_tools $ cd ./open_model_zoo/tools/downloader/
pi@raspberrypi:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader $ python downloader.py --name alexnet
################|| Downloading models ||################

========== Downloading /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/public/alexnet/alexnet.prototxt
... 100%, 3 KB, 9485 KB/s, 0 seconds passed

========== Downloading /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/public/alexnet/alexnet.caffemodel
... 100%, 238146 KB, 5054 KB/s, 47 seconds passed

################|| Post-processing ||################

========== Replacing text in /opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader/public/alexnet/alexnet.prototxt

Three files including a large model file alexnet.caffemodel has been downloaded.

pi@raspberrypi:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader $ ll public/alexnet/
total 238160
-rw-r--r-- 1 pi pi 243862414 Jan  8 20:17 alexnet.caffemodel
-rw-r--r-- 1 pi pi      3628 Jan  8 20:17 alexnet.prototxt
-rw-r--r-- 1 pi pi      3629 Jan  8 16:32 alexnet.prototxt.orig

3.3.2.3 Model Optimization

We then need to optimize the downloaded caffe model and make it feedable to OpenVINO.

pi@raspberrypi:/opt/intel/openvino/deployment_tools/open_model_zoo/tools/downloader $ cd ../../../model_optimizer/
pi@raspberrypi:/opt/intel/openvino/deployment_tools/model_optimizer $ python mo.py --input_model ../open_model_zoo/tools/downloader/public/alexnet/alexnet.caffemodel
Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/opt/intel/openvino/deployment_tools/model_optimizer/../open_model_zoo/tools/downloader/public/alexnet/alexnet.caffemodel
	- Path for generated IR: 	/opt/intel/openvino/deployment_tools/model_optimizer/.
	- IR output name: 	alexnet
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	False
Caffe specific parameters:
	- Path to Python Caffe* parser generated from caffe.proto: 	mo/front/caffe/proto
	- Enable resnet optimization: 	True
	- Path to the Input prototxt: 	/opt/intel/openvino/deployment_tools/model_optimizer/../open_model_zoo/tools/downloader/public/alexnet/alexnet.prototxt
	- Path to CustomLayersMapping.xml: 	Default
	- Path to a mean file: 	Not specified
	- Offsets for a mean file: 	Not specified
Model Optimizer version: 	2019.3.0-408-gac8584cb7
Please expect that Model Optimizer conversion might be slow. You are currently using Python protobuf library implementation. 
However you can use the C++ protobuf implementation that is supplied with the OpenVINO toolkitor build protobuf library from sources. 
Navigate to "install_prerequisites" folder and run: python -m easy_install protobuf-3.5.1-py($your_python_version)-win-amd64.egg 
set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp


 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #80. 

[ SUCCESS ] Generated IR model.
[ SUCCESS ] XML file: /opt/intel/openvino/deployment_tools/model_optimizer/./alexnet.xml
[ SUCCESS ] BIN file: /opt/intel/openvino/deployment_tools/model_optimizer/./alexnet.bin
[ SUCCESS ] Total execution time: 676.37 seconds.

Note: You may meet the following ERRORs during model optimization.

[ ERROR ]  
Detected not satisfied dependencies:
	networkx: installed: 2.4, required: 2.4
	protobuf: installed: 3.11.2, required: 3.6.1

Clearly, for networkx, the ERROR message is a kind of ridiculous.
Anyway, if you meet the above 2 errors, please DOWNGRADE your packages as follows:

pi@raspberrypi:/opt/intel/openvino/deployment_tools/model_optimizer $ pip install protobuf==3.6.1 --user
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting protobuf==3.6.1
  Downloading https://files.pythonhosted.org/packages/77/78/a7f1ce761e2c738e209857175cd4f90a8562d1bde32868a8cd5290d58926/protobuf-3.6.1-py2.py3-none-any.whl (390kB)
     |████████████████████████████████| 399kB 1.4MB/s 
Requirement already satisfied: six>=1.9 in /home/pi/.local/lib/python3.7/site-packages (from protobuf==3.6.1) (1.13.0)
Requirement already satisfied: setuptools in /home/pi/.local/lib/python3.7/site-packages (from protobuf==3.6.1) (44.0.0)
Installing collected packages: protobuf
  Found existing installation: protobuf 3.11.2
    Uninstalling protobuf-3.11.2:
      Successfully uninstalled protobuf-3.11.2
Successfully installed protobuf-3.6.1
pi@raspberrypi:/opt/intel/openvino/deployment_tools/model_optimizer $ pip install networkx==2.3 --user
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting networkx==2.3
  Downloading https://files.pythonhosted.org/packages/85/08/f20aef11d4c343b557e5de6b9548761811eb16e438cee3d32b1c66c8566b/networkx-2.3.zip (1.7MB)
     |████████████████████████████████| 1.8MB 2.3MB/s 
Requirement already satisfied: decorator>=4.3.0 in /home/pi/.local/lib/python3.7/site-packages (from networkx==2.3) (4.4.1)
Building wheels for collected packages: networkx
  Building wheel for networkx (setup.py) ... done
  Created wheel for networkx: filename=networkx-2.3-py2.py3-none-any.whl size=1556408 sha256=9964dc8e3b41e97f0228afb2c1c6ca40a5dfc07dc1a2574b92717114f3d5e533
  Stored in directory: /home/pi/.cache/pip/wheels/de/63/64/3699be2a9d0ccdb37c7f16329acf3863fd76eda58c39c737af
Successfully built networkx
Installing collected packages: networkx
  Found existing installation: networkx 2.4
    Uninstalling networkx-2.4:
      Successfully uninstalled networkx-2.4
Successfully installed networkx-2.3

3.3.2.4 Run The Sample

pi@raspberrypi:/opt/intel/openvino/deployment_tools/inference_engine/samples/build $ ./armv7l/Release/classification_sample_async -i ./me.jpg -m /opt/intel/openvino/deployment_tools/model_optimizer/alexnet.xml -nt 5 -d HETERO:MYRIAD
[ INFO ] InferenceEngine: 
	API version ............ 2.1
	Build .................. custom_releases/2019/R3_cb6cad9663aea3d282e0e8b3e0bf359df665d5d0
	Description ....... API
[ INFO ] Parsing input parameters
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     ./me.jpg
[ INFO ] Creating Inference Engine
	HETERO
	heteroPlugin version ......... 2.1
	Build ........... heteroPlugin
	MYRIAD
	myriadPlugin version ......... 2.1
	Build ........... 30677

[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ WARNING ] Image is resized from (924, 1280) to (227, 227)
[ INFO ] Batch size is 1
[ INFO ] Loading model to the device
[ INFO ] Create infer request
[ INFO ] Start inference (10 asynchronous executions)
[ INFO ] Completed 1 async request execution
[ INFO ] Completed 2 async request execution
[ INFO ] Completed 3 async request execution
[ INFO ] Completed 4 async request execution
[ INFO ] Completed 5 async request execution
[ INFO ] Completed 6 async request execution
[ INFO ] Completed 7 async request execution
[ INFO ] Completed 8 async request execution
[ INFO ] Completed 9 async request execution
[ INFO ] Completed 10 async request execution
[ INFO ] Processing output blobs

Top 5 results:

Image ./me.jpg

classid probability
------- -----------
838     0.0751343  
978     0.0728149  
977     0.0722656  
975     0.0606079  
903     0.0533142  

[ INFO ] Execution successful

[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool

4. A Leftover Bug

OpenCV dnn still fails to be built so far.

pi@raspberrypi:/opt/intel/openvino/inference_engine/samples/build $ python openvino_fd_myriad.py 
Traceback (most recent call last):
  File "openvino_fd_myriad.py", line 3, in <module>
    net = cv.dnn.readNet('face-detection-adas-0001.xml',
AttributeError: module 'cv2' has no attribute 'dnn'

Detectron2

Posted on 2019-10-13 | Edited on 2019-10-15 | In AI , DeepLearning

The autumn in both Vancouver and Seattle is gorgeous…

Overview Seattle on Space Needle	Around Space Needle

Space Needle	So Heavy

Ferris Wheel	Pier 55

Vancouver Maple	UBC Poisonous Mushroom

Alright, let’s rapidly test Detectron2.

Installation is detailedly summarized in INSTALL.md.

We can simply follow GETTING_STARTED.md for some simple demonstrations. Make sure you’ve downloaded the demo pictures from Detectron1 demo and save under Detectron2’s folder demo.

longervision-GT72-6QE% python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \
  --input ./demo/17790319373_bd19b24cfc_k.jpg \
  --output ./result.jpg \
  --opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl
[10/13 17:47:01 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml', input=['./demo/17790319373_bd19b24cfc_k.jpg'], opts=['MODEL.WEIGHTS', 'detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl'], output='./result.jpg', video_input=None, webcam=False)
WARNING [10/13 17:47:01 d2.config.compat]: Config 'configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
  0%|                                                                                                                                                                                        | 0/1 [00:00<?, ?it/s][10/13 17:47:05 detectron2]: ./demo/17790319373_bd19b24cfc_k.jpg: detected 13 instances in 0.29s
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.59it/s]

Let’s take a look at the result:

17790319373_bd19b24cfc_k

And of course, my pictures taken in Seattle and Vancouver:

Overview Seattle on Space Needle	Around Space Needle

Space Needle	So Heavy

Ferris Wheel	Pier 55

Oooooops… it seems there are still quite a lot of ERROR detections…
Anyway… tomorrow, 热干面. Google translator: Hot Noodles with Sesame Paste…

Tensorflow 2.0

Posted on 2019-10-12 | In AI , DeepLearning

Flying back to China soon. Before leaving, here comes a simple blog testing Tensorflow 2.0. In this blog, I strictly follow Amita Kapoor and Ajit Jaokar’s free book Getting Started with TensorFlow 2.0.

For simplicity, let’s try it out directly:

1. Tensorflow 2.0

2. Tensorflow Dataset

Build Toolchain Using Crosstool-NG

Posted on 2019-09-06 | Edited on 2019-09-09 | In Operating System

Today, we’re going to build our own toolchain using crosstool-NG. There are so many cases that we want to build a particular toolchain for a specific embedded system development board. One MOST important reason for that is probably because the specific development board has limited resource to build upon, therefore, the time-consumption to build some software on the development board is VERY inefficient. In the other way around, cross compiling on the host PC is MORE efficient with respective to the time-consumption.

In this blog, for simplicity, we take Raspberry Pi 3B as our demo development board, for which we are building the cross compiler. Some references can be found at:

1. Installation

How to install crosstool-NG are thoroughly summarized at its official website. In my case, I had it installed under folder /opt/compilers/crosstool-ng. Let’s take a look:

longervision-GT72-6QE% pwd
/opt/compilers/crosstool-ng
longervision-GT72-6QE% ls
bin  libexec  share
longervision-GT72-6QE% ls bin
ct-ng

And make sure /opt/compilers/crosstool-ng/bin is under environment variable PATH.

2. Configuration

Under any directory that you want to save your .config file, we can configure our target cross compiler.

1	longervision-GT72-6QE% ct-ng menuconfig

Crosstool-NG

According to elinux: RPi Linaro GCC Compilation, we need to do the following selections:

Paths and misc options:
- Try features marked as EXPERIMENTAL: ticked
- Prefix directory: input the full path where you want to save the built toolchains
- Number of parallel jobs: 4. Sorry that I’ve NO idea if this is the number of cores on Raspberry Pi 3B, but Raspberry Pi 3B does have 4 cores.
Target options:
- Target Architecture: arm
- Default instruction set mode: arm
- Use EABI: ticked
- append hf to the tuple (EXPERIMENTAL): ticked
- Endianness: Little endian
- Bitness: 32-bit
- Emit assembly for CPU: cortex-a53
- Use specific FPU: vfp
- Floating point: hardware (FPU)
Toolchain options:
- Tuple’s vendor string: rpi
Operating System
- Target OS: linux
- Version of Linux: 4.20.8. Currently, I flashed Raspbian Buster with desktop and recommended software, 2019-07-10 onto my Raspberry Pi 3B, it comes with kernel of version 4.19.66, which is NOT in the list. That’s why I decided to select 4.20.8 and try my luck.
1
2
➜ ~ uname -a
Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

Binary utilities:

Binary format: ELF
Version of binutils: 2.31.1

➜  ~ apt show binutils
    Package: binutils
    Version: 2.31.1-16+rpi1
    Priority: optional
    Section: devel
    Maintainer: Matthias Klose <doko@debian.org>
    Installed-Size: 95.2 kB
    Provides: binutils-gold, elf-binutils
    Depends: binutils-common (= 2.31.1-16+rpi1), libbinutils (= 2.31.1-16+rpi1), binutils-arm-linux-gnueabihf (= 2.31.1-16+rpi1)
    Suggests: binutils-doc (>= 2.31.1-16+rpi1)
    Conflicts: binutils-multiarch (<< 2.27-8), modutils (<< 2.4.19-1)
    Homepage: https://www.gnu.org/software/binutils/
    Download-Size: 56.9 kB
    APT-Manual-Installed: no
    APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
    Description: GNU assembler, linker and binary utilities
    The programs in this package are used to assemble, link and manipulate
    binary and object files.  They may be used in conjunction with a compiler
    and various libraries to build programs.

C-library:

C library: glibc
Version of glibc: 2.28

➜  ~ apt show libc-bin
    Package: libc-bin
    Version: 2.28-10+rpi1
    Priority: required
    Essential: yes
    Section: libs
    Source: glibc
    Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
    Installed-Size: 3,015 kB
    Depends: libc6 (>> 2.28), libc6 (<< 2.29)
    Recommends: manpages
    Homepage: https://www.gnu.org/software/libc/libc.html
    Download-Size: 657 kB
    APT-Manual-Installed: yes
    APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
    Description: GNU C Library: Binaries
    This package contains utility programs related to the GNU C Library.
    .
    * catchsegv: catch segmentation faults in programs
    * getconf: query system configuration variables
    * getent: get entries from administrative databases
    * iconv, iconvconfig: convert between character encodings
    * ldd, ldconfig: print/configure shared library dependencies
    * locale, localedef: show/generate locale definitions
    * tzselect, zdump, zic: select/dump/compile time zones

C compiler:

Show gcc versions from: GNU
Version of gcc: 8.3.0
gcc extra config: --with-float=hard
Link libstdc++ statically into the gcc binary: tick
C++: tick

➜  ~ apt show gcc
Package: gcc
Version: 4:8.3.0-1+rpi2
Priority: optional
Section: devel
Source: gcc-defaults (1.181+rpi2)
Maintainer: Debian GCC Maintainers <debian-gcc@lists.debian.org>
Installed-Size: 46.1 kB
Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
Recommends: libc6-dev | libc-dev
Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
Conflicts: gcc-doc (<< 1:2.95.3)
Download-Size: 5,200 B
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C compiler
This is the GNU C compiler, a fairly portable optimizing compiler for C.
.
This is a dependency package providing the default GNU C compiler.

After we save the configuration to file .config, we Exit the crosstool-NG configuration dialog.

3. Build

Please remember to:

unset LD_LIBRARY_PATH before building. Otherwise, you’ll meet some ERROR messages.
mkdir ~/src before building. Otherwise, whenever you tried to rerun ct-ng build, you’ll have to download ALL required packages from scratch.

longervision-GT72-6QE% ct-ng build
[INFO ]  Performing some trivial sanity checks
[WARN ]  Number of open files 1024 may not be sufficient to build the toolchain; increasing to 2048
[INFO ]  Build started 20190906.014128
[INFO ]  Building environment variables
[EXTRA]  Preparing working directories
[EXTRA]  Installing user-supplied crosstool-NG configuration
[EXTRA]  =================================================================
[EXTRA]  Dumping internal crosstool-NG configuration
[EXTRA]    Building a toolchain for:
[EXTRA]      build  = x86_64-pc-linux-gnu
[EXTRA]      host   = x86_64-pc-linux-gnu
[EXTRA]      target = arm-rpi-linux-gnueabihf
[EXTRA]  Dumping internal crosstool-NG configuration: done in 0.07s (at 00:01)
[INFO ]  =================================================================
[INFO ]  Retrieving needed toolchain components' tarballs
[EXTRA]    Retrieving 'ncurses-6.1'
[EXTRA]    Verifying SHA512 checksum for 'ncurses-6.1.tar.gz'
[EXTRA]    Saving 'ncurses-6.1.tar.gz' to local storage
[EXTRA]    Retrieving 'libiconv-1.15'
[EXTRA]    Verifying SHA512 checksum for 'libiconv-1.15.tar.gz'
[EXTRA]    Saving 'libiconv-1.15.tar.gz' to local storage
[EXTRA]    Retrieving 'gettext-0.19.8.1'
[EXTRA]    Verifying SHA512 checksum for 'gettext-0.19.8.1.tar.xz'
[EXTRA]    Saving 'gettext-0.19.8.1.tar.xz' to local storage
[EXTRA]    Retrieving 'binutils-2.31.1'
[EXTRA]    Verifying SHA512 checksum for 'binutils-2.31.1.tar.xz'
[EXTRA]    Saving 'binutils-2.31.1.tar.xz' to local storage
[EXTRA]    Retrieving 'gcc-8.3.0'
[EXTRA]    Verifying SHA512 checksum for 'gcc-8.3.0.tar.xz'
[EXTRA]    Saving 'gcc-8.3.0.tar.xz' to local storage
[EXTRA]    Retrieving 'glibc-2.28'
[EXTRA]    Verifying SHA512 checksum for 'glibc-2.28.tar.xz'
[EXTRA]    Saving 'glibc-2.28.tar.xz' to local storage
[INFO ]  Retrieving needed toolchain components' tarballs: done in 130.35s (at 02:12)
[INFO ]  =================================================================
[INFO ]  Extracting and patching toolchain components
[EXTRA]    Extracting linux-4.20.8
[EXTRA]    Patching linux-4.20.8
[EXTRA]    Extracting zlib-1.2.11
[EXTRA]    Patching zlib-1.2.11
[EXTRA]    Extracting gmp-6.1.2
[EXTRA]    Patching gmp-6.1.2
[EXTRA]    Extracting mpfr-4.0.2
[EXTRA]    Patching mpfr-4.0.2
[EXTRA]    Extracting isl-0.20
[EXTRA]    Patching isl-0.20
[EXTRA]    Extracting mpc-1.1.0
[EXTRA]    Patching mpc-1.1.0
[EXTRA]    Extracting ncurses-6.1
[EXTRA]    Patching ncurses-6.1
[EXTRA]    Extracting libiconv-1.15
[EXTRA]    Patching libiconv-1.15
[EXTRA]    Extracting gettext-0.19.8.1
[EXTRA]    Patching gettext-0.19.8.1
[EXTRA]    Extracting binutils-2.31.1
[EXTRA]    Patching binutils-2.31.1
[EXTRA]    Extracting gcc-8.3.0
[EXTRA]    Patching gcc-8.3.0
[EXTRA]    Extracting glibc-2.28
[EXTRA]    Patching glibc-2.28
[INFO ]  Extracting and patching toolchain components: done in 33.39s (at 02:45)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for build
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for build: done in 14.62s (at 03:00)
[INFO ]  =================================================================
[INFO ]  Installing zlib for host
[EXTRA]    Configuring zlib
[EXTRA]    Building zlib
[EXTRA]    Installing zlib
[INFO ]  Installing zlib for host: done in 1.12s (at 03:01)
[INFO ]  =================================================================
[INFO ]  Installing GMP for host
[EXTRA]    Configuring GMP
[EXTRA]    Building GMP
[EXTRA]    Installing GMP
[INFO ]  Installing GMP for host: done in 30.12s (at 03:31)
[INFO ]  =================================================================
[INFO ]  Installing MPFR for host
[EXTRA]    Configuring MPFR
[EXTRA]    Building MPFR
[EXTRA]    Installing MPFR
[INFO ]  Installing MPFR for host: done in 17.35s (at 03:48)
[INFO ]  =================================================================
[INFO ]  Installing ISL for host
[EXTRA]    Configuring ISL
[EXTRA]    Building ISL
[EXTRA]    Installing ISL
[INFO ]  Installing ISL for host: done in 17.73s (at 04:06)
[INFO ]  =================================================================
[INFO ]  Installing MPC for host
[EXTRA]    Configuring MPC
[EXTRA]    Building MPC
[EXTRA]    Installing MPC
[INFO ]  Installing MPC for host: done in 5.47s (at 04:12)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for host
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for host: done in 13.64s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing libiconv for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing libiconv for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing gettext for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing gettext for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing binutils for host
[EXTRA]    Configuring binutils
[EXTRA]    Building binutils
[EXTRA]    Installing binutils
[INFO ]  Installing binutils for host: done in 46.89s (at 05:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-1 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-1 core C gcc compiler: done in 285.36s (at 09:58)
[INFO ]  =================================================================
[INFO ]  Installing kernel headers
[EXTRA]    Installing kernel headers
[EXTRA]    Checking installed headers
[INFO ]  Installing kernel headers: done in 6.47s (at 10:04)
[INFO ]  =================================================================
[INFO ]  Installing C library headers & start files
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Configuring C library
[EXTRA]      Installing C library headers
[EXTRA]      Installing C library start files
[INFO ]    Building for multilib 1/1: '': done in 7.65s (at 10:12)
[INFO ]  Installing C library headers & start files: done in 7.72s (at 10:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-2 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-2 core C gcc compiler: done in 354.87s (at 16:07)
[INFO ]  =================================================================
[INFO ]  Installing C library
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Cleaning up start files
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Installing C library
[INFO ]    Building for multilib 1/1: '': done in 184.69s (at 19:12)
[INFO ]  Installing C library: done in 184.78s (at 19:12)
[INFO ]  =================================================================
[INFO ]  Installing final gcc compiler
[EXTRA]    Configuring final gcc compiler
[EXTRA]    Building final gcc compiler
[EXTRA]    Installing final gcc compiler
[EXTRA]    Housekeeping for final gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing final gcc compiler: done in 339.13s (at 24:51)
[INFO ]  =================================================================
[INFO ]  Finalizing the toolchain's directory
[INFO ]    Stripping all toolchain executables
[EXTRA]    Installing the populate helper
[EXTRA]    Installing a cross-ldd helper
[EXTRA]    Creating toolchain aliases
[EXTRA]    Removing installed documentation
[EXTRA]    Collect license information from: ~/Crosscompile/.build/arm-rpi-linux-gnueabihf/src
[EXTRA]    Put the license information to: ~/....../RPi/share/licenses
[INFO ]  Finalizing the toolchain's directory: done in 1.80s (at 24:52)
[INFO ]  Build completed at 20190906.020619
[INFO ]  (elapsed: 24:51.86)
[INFO ]  Finishing installation (may take a few seconds)...
[24:52] / %                                                                                                                                                              longervision-GT72-6QE%

This process may take a while. I’m going to sleep tonight. Continue tomorrow…

Alright, let’s continue today. Glad to know it’s built successfully.

Let’s FIRST take a look at what’s under the current folder.

1 2	longervision-GT72-6QE% ls build.log rpi.config

And then, let’s take a look at what’s built under the specified destination folder.

longervision-GT72-6QE% ls ~/....../CrossCompile/RPi
arm-rpi-linux-gnueabihf  bin  build.log.bz2  include  lib  libexec  share
longervision-GT72-6QE% ls ~/....../CrossCompile/RPi/bin
arm-rpi-linux-gnueabihf-addr2line  arm-rpi-linux-gnueabihf-cpp           arm-rpi-linux-gnueabihf-gcc-ar      arm-rpi-linux-gnueabihf-gprof    arm-rpi-linux-gnueabihf-objdump   arm-rpi-linux-gnueabihf-strip
arm-rpi-linux-gnueabihf-ar         arm-rpi-linux-gnueabihf-ct-ng.config  arm-rpi-linux-gnueabihf-gcc-nm      arm-rpi-linux-gnueabihf-ld       arm-rpi-linux-gnueabihf-populate
arm-rpi-linux-gnueabihf-as         arm-rpi-linux-gnueabihf-elfedit       arm-rpi-linux-gnueabihf-gcc-ranlib  arm-rpi-linux-gnueabihf-ld.bfd   arm-rpi-linux-gnueabihf-ranlib
arm-rpi-linux-gnueabihf-c++        arm-rpi-linux-gnueabihf-g++           arm-rpi-linux-gnueabihf-gcov        arm-rpi-linux-gnueabihf-ldd      arm-rpi-linux-gnueabihf-readelf
arm-rpi-linux-gnueabihf-cc         arm-rpi-linux-gnueabihf-gcc           arm-rpi-linux-gnueabihf-gcov-dump   arm-rpi-linux-gnueabihf-nm       arm-rpi-linux-gnueabihf-size
arm-rpi-linux-gnueabihf-c++filt    arm-rpi-linux-gnueabihf-gcc-8.3.0     arm-rpi-linux-gnueabihf-gcov-tool   arm-rpi-linux-gnueabihf-objcopy  arm-rpi-linux-gnueabihf-strings
longervision-GT72-6QE%

Finally, let’s take a look at the version of our built cross compilers for Raspberry Pi 3B.

longervision-GT72-6QE% arm-rpi-linux-gnueabihf-gcc --version
zsh: command not found: arm-rpi-linux-gnueabihf-gcc
longervision-GT72-6QE% ./arm-rpi-linux-gnueabihf-gcc --version
arm-rpi-linux-gnueabihf-gcc (crosstool-NG 1.24.0.6-afaf7b9) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

longervision-GT72-6QE% ./arm-rpi-linux-gnueabihf-g++ --version
arm-rpi-linux-gnueabihf-g++ (crosstool-NG 1.24.0.6-afaf7b9) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Additional issue: It seems current crosstool-NG does NOT officially support Python. Please refer to my issue at crosstool-NG

4. Compile/Build a Package with Generated Toolchain

For most of the packages nowadays, they are

either supported by make: ./configure -> make -> make install
or supported by CMake: mkdir build -> cd build -> ccmake ../ -> make -> make install

How to use our generated toolchain to compile/build our target packages?

For the FIRST option, you can follow the crosstool-NG Using the toolchain.
For the SECOND option, you are welcome to follow CMake Toolchains

Today, we’re going to take the package flann as an example, which are to be built with CMake.

4.1 CMakeLists.txt Modification

4.1.1 CMake Toolchains - Cross Compiling for Linux

By following CMake Toolchains - Cross Compiling for Linux, we FIRST modify flann CMakeLists.txt a bit by adding the following lines before project(flann).

cmake_minimum_required(VERSION 2.6)

if(COMMAND cmake_policy)
    cmake_policy(SET CMP0003 NEW)
endif(COMMAND cmake_policy)

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)

set(CMAKE_SYSROOT ....../arm-rpi-linux-gnueabihf/sysroot)
set(CMAKE_STAGING_PREFIX ....../stage)

set(tools /opt/compilers/RPi)
set(CMAKE_C_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-g++)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")

project(flann)
string(TOLOWER ${PROJECT_NAME} PROJECT_NAME_LOWER)

CMAKE_SYSROOT specify the sysroot directory which emulate your target environment, here, Raspberry Pi 3B
CMAKE_STAGING_PREFIX is where the STAGING results store, for the reason that final results may require to be built in multiple stages. You may refer to Linux From Scratch for further background knowledge about that.
tools actually specify the building tool directory. Making sure all generated cross compiling tools are under folder ${tools}/bin.

4.1.2 Ignore hdf5

In addition, for the emulated Raspberry Pi 3B sysroot, hdf5 is NOT supported. Therefore, let’s simply comment out the following line in flann CMakeLists.txt.

1	#find_hdf5()

4.2 Cross Compile

Now, let’s start cross-compiling flann.

1
2
3

longervision-GT72-6QE% mkdir build
longervision-GT72-6QE% cd build
longervision-GT72-6QE% ccmake ../

and press c and then t, you’ll see the cross compiling toolchains have been automatically configured as:

BUILD_CUDA_LIB                  *OFF
BUILD_C_BINDINGS                *ON
BUILD_DOC                       *OFF
BUILD_EXAMPLES                  *OFF
BUILD_MATLAB_BINDINGS           *OFF
BUILD_PYTHON_BINDINGS           *ON
BUILD_TESTS                     *OFF
CMAKE_AR                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar
CMAKE_BUILD_TYPE                *RelWithDebInfo
CMAKE_COLOR_MAKEFILE            *ON
CMAKE_CXX_COMPILER_AR           */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_CXX_COMPILER_RANLIB       */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_CXX_FLAGS                 *-I/usr/include
CMAKE_CXX_FLAGS_DEBUG           *-g
CMAKE_CXX_FLAGS_MINSIZEREL      *-Os -DNDEBUG
CMAKE_CXX_FLAGS_RELEASE         *-O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO  *-O2 -g -DNDEBUG
CMAKE_C_COMPILER_AR             */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_C_COMPILER_RANLIB         */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_C_FLAGS                   *-I/usr/include
CMAKE_C_FLAGS_DEBUG             *-g
CMAKE_C_FLAGS_MINSIZEREL        *-Os -DNDEBUG
CMAKE_C_FLAGS_RELEASE           *-O3 -DNDEBUG
CMAKE_C_FLAGS_RELWITHDEBINFO    *-O2 -g -DNDEBUG
CMAKE_EXE_LINKER_FLAGS          *
CMAKE_EXE_LINKER_FLAGS_DEBUG    *
CMAKE_EXE_LINKER_FLAGS_MINSIZE  *  
CMAKE_EXE_LINKER_FLAGS_RELEASE  *     
CMAKE_EXE_LINKER_FLAGS_RELWITH  *
CMAKE_EXPORT_COMPILE_COMMANDS   *OFF
CMAKE_INSTALL_PREFIX            */usr/local
CMAKE_LINKER                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ld
CMAKE_MAKE_PROGRAM              */usr/bin/make
CMAKE_MODULE_LINKER_FLAGS       *    
CMAKE_MODULE_LINKER_FLAGS_DEBU  *  
CMAKE_MODULE_LINKER_FLAGS_MINS  *   
CMAKE_MODULE_LINKER_FLAGS_RELE  *    
CMAKE_MODULE_LINKER_FLAGS_RELW  * 
CMAKE_NM                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-nm
CMAKE_OBJCOPY                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objcopy
CMAKE_OBJDUMP                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objdump
CMAKE_RANLIB                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib
CMAKE_SHARED_LINKER_FLAGS       *    
CMAKE_SHARED_LINKER_FLAGS_DEBU  *
CMAKE_SHARED_LINKER_FLAGS_MINS  *   
CMAKE_SHARED_LINKER_FLAGS_RELE  *
CMAKE_SHARED_LINKER_FLAGS_RELW  *   
CMAKE_SKIP_INSTALL_RPATH        *OFF
CMAKE_SKIP_RPATH                *OFF
CMAKE_STATIC_LINKER_FLAGS       *
CMAKE_STATIC_LINKER_FLAGS_DEBU  *
CMAKE_STATIC_LINKER_FLAGS_MINS  * 
CMAKE_STATIC_LINKER_FLAGS_RELE  *
CMAKE_STATIC_LINKER_FLAGS_RELW  *
CMAKE_STRIP                     */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-strip
CMAKE_VERBOSE_MAKEFILE          *ON
CPACK_SOURCE_RPM                *OFF
CPACK_SOURCE_TBZ2               *ON
CPACK_SOURCE_TGZ                *ON
CPACK_SOURCE_TXZ                *ON 
CPACK_SOURCE_TZ                 *ON
CPACK_SOURCE_ZIP                *OFF
DPKG_PROGRAM                    */usr/bin/dpkg
NSIS_PROGRAM                    */usr/bin/makensis
NVCC_COMPILER_BINDIR            *
PACKAGE_MAKER_PROGRAM           *PACKAGE_MAKER_PROGRAM-NOTFOUND
PKG_CONFIG_EXECUTABLE           */usr/bin/pkg-config
PYTHON_EXECUTABLE               */usr/bin/python
RPM_PROGRAM                     *RPM_PROGRAM-NOTFOUND
USE_MPI                         *OFF
USE_OPENMP                      *OFF

The LAST step before make is to modify some of the parameters accordingly, in my case:

BUILD_PYTHON_BINDINGS: ON -> OFF
CMAKE_CXX_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
CMAKE_C_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
CMAKE_VERBOSE_MAKEFILE: OFF -> ON
CMAKE_INSTALL_PREFIX: /usr/local/ -> under which directory you want to install, which can be IGNORED for now

Now, press c, you’ll see

1 2	CMake Warning at CMakeLists.txt:99 (message): hdf5 library not found, some tests will not be run

A warining of missing hdf5 is clearly reasonable and acceptable. Then press g.

Finally, it’s the time for us to cross build flann.

longervision-GT72-6QE% make -j8
/usr/bin/cmake -S....../flann -B....../flann/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles ....../flann/build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/depend
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_s.dir/DependInfo.cmake --color=
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp_s.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp_s
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/build
make[2]: Entering directory '....../flann/build'
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/build
make[2]: Entering directory '....../flann/build'
[ 12%] Building CXX object src/cpp/CMakeFiles/flann_s.dir/flann/flann.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_s.dir/flann/flann.cpp.o -c ....../flann/src/cpp/flann/flann.cpp
[ 25%] Building CXX object src/cpp/CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -DFLANN_USE_CUDA -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o -c ....../flann/src/cpp/flann/flann_cpp.cpp
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann_cpp.cpp:30:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
[ 37%] Linking CXX static library ../../lib/libflann_cpp_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_cpp_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_cpp_s.a  CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_cpp_s.a
make[2]: Leaving directory '....../flann/build'
[ 37%] Built target flann_cpp_s
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/build
make[2]: Entering directory '....../flann/build'
[ 50%] Building CXX object src/cpp/CMakeFiles/flann_cpp.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann_cpp.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[ 62%] Linking CXX shared library ../../lib/libflann_cpp.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann_cpp.so.1.9 -o ../../lib/libflann_cpp.so.1.9.1 CMakeFiles/flann_cpp.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_cpp_s.a -Wl,-no-whole-archive 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann_cpp.so.1.9.1 ../../lib/libflann_cpp.so.1.9 ../../lib/libflann_cpp.so
make[2]: Leaving directory '....../flann/build'
[ 62%] Built target flann_cpp
In file included from ....../flann/src/cpp/flann/algorithms/all_indices.h:35,
                 from ....../flann/src/cpp/flann/flann.hpp:45,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:

VERY Important:

If you build flann from source on a Raspberry Pi 3B, your system is going to hang HERE, possibly due to lack of memory..
Raspberry Pi 3B has 4 cores ONLY, but now you can use MORE cores on your host PC, which is clearly one advantage of using cross compiling.

Finally, you’ll see flann has been successfully cross compiled as:

....../flann/src/cpp/flann/algorithms/autotuned_index.h: In member function 'void flann::AutotunedIndex<Distance>::optimizeKMeans(std::vector<flann::AutotunedIndex<Distance>::CostData>&) [with Distance = flann::L2<double>]':
....../flann/src/cpp/flann/algorithms/autotuned_index.h:379:31: note: parameter passing for argument of type 'flann::L2<double>' changed in GCC 7.1
         KMeansIndex<Distance> kmeans(sampledDataset_, cost.params, distance_);
                               ^~~~~~
[ 75%] Linking CXX static library ../../lib/libflann_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_s.a  CMakeFiles/flann_s.dir/flann/flann.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_s.a
make[2]: Leaving directory '....../flann/build'
[ 75%] Built target flann_s
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann.dir/DependInfo.cmake --color=
Scanning dependencies of target flann
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/build
make[2]: Entering directory '....../flann/build'
[ 87%] Building CXX object src/cpp/CMakeFiles/flann.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[100%] Linking CXX shared library ../../lib/libflann.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann.so.1.9 -o ../../lib/libflann.so.1.9.1 CMakeFiles/flann.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_s.a -Wl,-no-whole-archive -Wl,--push-state,--no-as-needed -Wl,--pop-state 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann.so.1.9.1 ../../lib/libflann.so.1.9 ../../lib/libflann.so
make[2]: Leaving directory '....../flann/build'
[100%] Built target flann
make[1]: Leaving directory '....../flann/build'
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles 0

You can now:

make install to install the built/generated libraries installed under CMAKE_INSTALL_PREFIX
copy and paste the built/generated libraries onto Raspberry Pi 3B and use it directly.

BTW: do NOT forget to install the header files.

5. Additional Issues

Multilib/multiarch seems to be problematic nowadays. Please pay attention to Multilib/multiarch. Some related issues are enumuated as the end of this blog.

Alright, that’s all for today. Let me go to bed. Good bye…

Stereo Vision on VCSBC nano Z-RH-2 - PART I

Posted on 2019-08-30 | Edited on 2019-09-18 | In Machine Vision , FPGA

Today, we are going to talk about a fabulous project: stereo vision on a zynq-7010 board.

1. VCSBC nano Z-RH-2

1.1 Hardware

We are using a VCSBC nano Z-RH-2 board for today’s experiment. The board adopted looks like the following:

Front	Back	Connector

More detailed specifications, please refer to Vision Components’s official website.

1.2 Software

After you set up a static IP for this Vision Components SBC, it’s pretty straightforward to ssh into the system.

longervision-GT72-6QE% ssh user@192.168.1.79
user@192.168.1.79's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Aug 29 18:17:49 2019 from 192.168.1.200
user@VC-Z:~$

Currently, VC provides Linux Kernel 3.14.79.

1 2	user@VC-Z:~$ uname -a Linux VC-Z 3.14.79-vc-z #2 SMP PREEMPT Wed Apr 24 18:33:06 CEST 2019 armv7l GNU/Linux

And, let’s take a look at the dual ARMv7 CPUs on zynq-7010.

user@VC-Z:~$ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 0 (v7l)
BogoMIPS	: 1725.23
Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpd32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x3
CPU part	: 0xc09
CPU revision	: 0

processor	: 1
model name	: ARMv7 Processor rev 0 (v7l)
BogoMIPS	: 1725.23
Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpd32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x3
CPU part	: 0xc09
CPU revision	: 0

Hardware	: Xilinx Zynq Platform
Revision	: 0000
Serial		: 0000000000000000

2. Stereo Vision

Sorry everybody. Today, I ONLY test stereo vision on ARM. I’ll try to figure out how to flash an open source IP of stereo vision onto zynq-7010, or write my own ASAP.

Hmmmmmmmm… It’s better I keep my code in dark???

2.1 Classical Image Pairs

Let’s try out the stereo vision on some .pgm image pairs FIRST.

user@VC-Z:~/longervision$ ./pgmpair ../images/aloe_left.pgm ../images/aloe_right.pgm 
width= 1282
height= 1110
fps: 0.119696
process: 398651

My GOD… It’s UNBELIEVABLY SLOW.

aloe_left	aloe_right

aloe_left Stereo	aloe_right Stereo

2.2 Live Video Pairs

The BEST demo code to test Vision Componenets Stereo Vision is Eclipse_Example_Projects_VC_Z.

2.2.1 imageCaptureTest

#./imageCaptureTest 
0


 ----------------------------------------------------------------------- 
  VCCapt @0xbe87153c                                    Sensor Count: 2  
 ----------------------------------------------------------------------- 
                                                                         
                  Camera Type :  NANO Z RH Rev.0                         
                Sensor 0 Type :  E2V_EV76C560ABT_1_0                     
                Sensor 1 Type :  E2V_EV76C560ABT_1_0                     
                                                                         
   Trigger        Source      :  IMM                                     
                                                                         
     ___________________________________________________________________ 
     Sensor[ 0]:         Shutter:     12345 us,  Gain: 0.00              
                                                                         
                   x0, y0       :  (    0,    0)                         
            binX,     binY      :  (    1,    1)                         
        subsmplX, subsmplY, flip:  (    1,    1,0x00)                    
    (img)          dx, dy, pitch:  ( 1280, 1024), 1280                   
                                                                         
    (img) st, ccmp1, ccmp2, type:  0xb6b03080, 0x00000000, 0x00000000, 0 
                                                                         
      Description                                                        
       Sen dx, Sen dy, Sen pitch:   1280, 1024, 1280                     
       Max dx, Max dy, Max pitch:   1280, 1024, 1280                     
    Binning Max (dx, dy), Square:      2,    1,    1                     
    Subsmpl Max (dx, dy), Square:      8,    8,    0                     
       Init       st,ccmp1,ccmp2:  0xb6b03080, 0x00000000, 0x00000000    
       Init  Shutter, Gain, Type:       10000,  0.00, GREY   (0)         
                  FD, Port, Type:  4, 0, E2V_EV76C560ABT_1_0             
                                                                         
      Volatile Description                                               
            requestCaptID, State:          0, IDLE                       
          captId, TimestampTicks:          0,   6476832867277            
                                                                         
      Externals Description                                              
            pos[0-3], angle[0-2]:  [ 0.00, 0.00, 0.00],[  0.0,  0.0]     
                                                                         
     ___________________________________________________________________ 
     Sensor[ 1]:         Shutter:     10000 us,  Gain: 0.00              
                                                                         
                   x0, y0       :  (    0,    0)                         
            binX,     binY      :  (    1,    1)                         
        subsmplX, subsmplY, flip:  (    1,    1,0x00)                    
    (img)          dx, dy, pitch:  ( 1280, 1024), 1280                   
                                                                         
    (img) st, ccmp1, ccmp2, type:  0xb69c2080, 0x00000000, 0x00000000, 0 
                                                                         
      Description                                                        
       Sen dx, Sen dy, Sen pitch:   1280, 1024, 1280                     
       Max dx, Max dy, Max pitch:   1280, 1024, 1280                     
    Binning Max (dx, dy), Square:      2,    1,    1                     
    Subsmpl Max (dx, dy), Square:      8,    8,    0                     
       Init       st,ccmp1,ccmp2:  0xb69c2080, 0x00000000, 0x00000000    
       Init  Shutter, Gain, Type:       10000,  0.00, GREY   (0)         
                  FD, Port, Type:  5, 1, E2V_EV76C560ABT_1_0             
                                                                         
      Volatile Description                                               
            requestCaptID, State:          0, IDLE                       
          captId, TimestampTicks:          0,   6476832867284            
                                                                         
      Externals Description                                              
            pos[0-3], angle[0-2]:  [ 0.00, 0.00, 0.00],[  0.0,  0.0]     
                                                                         
 ----------------------------------------------------------------------- 
......

2.2.2 imageCaptFPS

03:41:47[root@VC-Z] /home/user/vc/Eclipse_Example_Projects_VC_Z/imageCaptFPS
#./imageCaptFPS 
dy= 1024  Clock=100000000  Acquisiton&Copy Duration:         28s     780685us  for 1000 Cycles ==  34.745525fps.

dy= 768  Clock=100000000  Acquisiton&Copy Duration:         28s     789899us  for 1000 Cycles ==  34.734402fps.

dy= 640  Clock=100000000  Acquisiton&Copy Duration:         28s     790000us  for 1000 Cycles ==  34.734283fps.

dy= 512  Clock=100000000  Acquisiton&Copy Duration:         28s     790012us  for 1000 Cycles ==  34.734268fps.

dy= 384  Clock=100000000  Acquisiton&Copy Duration:         28s     790005us  for 1000 Cycles ==  34.734280fps.

dy= 256  Clock=100000000  Acquisiton&Copy Duration:         28s     790000us  for 1000 Cycles ==  34.734283fps.

dy= 192  Clock=100000000  Acquisiton&Copy Duration:         28s     790021us  for 1000 Cycles ==  34.734257fps.

dy= 128  Clock=100000000  Acquisiton&Copy Duration:         28s     789984us  for 1000 Cycles ==  34.734303fps.

dy= 64  Clock=100000000  Acquisiton&Copy Duration:         28s     789987us  for 1000 Cycles ==  34.734299fps.

dy= 32  Clock=100000000  Acquisiton&Copy Duration:         28s     789997us  for 1000 Cycles ==  34.734287fps.

dy= 16  Clock=100000000  Acquisiton&Copy Duration:         28s     790017us  for 1000 Cycles ==  34.734264fps.

dy= 8  Clock=100000000  Acquisiton&Copy Duration:         28s     790002us  for 1000 Cycles ==  34.734280fps.

dy= 4  Clock=100000000  Acquisiton&Copy Duration:         28s     789985us  for 1000 Cycles ==  34.734303fps.

dy= 2  Clock=100000000  Acquisiton&Copy Duration:         28s     789997us  for 1000 Cycles ==  34.734287fps.

dy= 1  Clock=100000000  Acquisiton&Copy Duration:         28s     789994us  for 1000 Cycles ==  34.734291fps.

The above 2 examples are directly run on Vision Components’s board without display, for the given cable is of VGA connector, which is ALREADY outdated for many years. Therefore, in order to show the captured image pairs, in the next section, we’ll have to stream the captured data to a host computer, and display the real-time video pairs.

2.3 Stream/Display From Host Computer

Due to Vision Componenets’ Official documentation VCLinux_Getting_Started.pdf, images captured from camera can be displayed on a remote hosting PC.

2.3.1 Eclipse_Example_Projects_VC_Z (Not Preferred)

Eclipse_Example_Projects_VC_Z.zip provides some source code, which displays images captured from camera on the hosting PC’s Eclipse, by adding the camera as a Remote System to Eclipse.
Anyway, by using this method, you need to prepare ALL the following software and packages in advance.

Eclipse
CDT: install by Check for Updates with p2 software repository: http://download.eclipse.org/tools/cdt/releases/9.8
Direct Remote C++ Debugging: installed from Eclipse Marketplace

2.3.2 vcimgnetclient.py

Besides the above method, a much more straightforward method is to adopt python script vcimgnetclient.py provided by Vision Components. However, vcimgnetclient.py is ONLY python2 compatible, and Vision Components has NO plan to provide a python3 compatible version of vcimgnetclient.py.

Therefore, the KEY to use the 2nd method is to make vcimgnetclient.py python3 compatible.

2.3.2.1 PyGTK to PyGTK3

1) 2to3

longervision-GT72-6QE% 2to3 vcimgnetclient.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored vcimgnetclient.py
--- vcimgnetclient.py	(original)
+++ vcimgnetclient.py	(refactored)
@@ -274,11 +274,11 @@
             if(chk_rgb_true):
                 dy /= 3
                 #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-                img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+                img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
             else:
                 dy /= 1
                 #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-            img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+            img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
             cpb = Gdk.pixbuf_new_from_data(buffer(img_data_packed), Gdk.COLORSPACE_RGB, False, 8, dx, dy, 3*pitch)
             cpb.save(path, "png")
             
@@ -589,16 +589,16 @@
                         dy /= 3
                         mv_img_data = memoryview(img['data'])
                         #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-                        img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+                        img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
                         self.area.window.draw_rgb_image(gc, 0,0, dx,dy, Gdk.RGB_DITHER_NONE, buffer(img_data_packed),3*pitch,0,0)
                     else:
                         if('flscol'==self.how_select):
                             mv_img_data = memoryview(img['data']).tolist()
-                            r_data = map(falsecolor_r,mv_img_data[0 * pitch * dy:])
-                            g_data = map(falsecolor_g,mv_img_data[0 * pitch * dy:])
-                            b_data = map(falsecolor_b,mv_img_data[0 * pitch * dy:])
+                            r_data = list(map(falsecolor_r,mv_img_data[0 * pitch * dy:]))
+                            g_data = list(map(falsecolor_g,mv_img_data[0 * pitch * dy:]))
+                            b_data = list(map(falsecolor_b,mv_img_data[0 * pitch * dy:]))
                             #img_data_packed = bytearray(chain.from_iterable(izip(r_data, g_data, b_data)))
-                            img_data_packed = bytearray(chain.from_iterable(zip(r_data, g_data, b_data)))
+                            img_data_packed = bytearray(chain.from_iterable(list(zip(r_data, g_data, b_data))))
                             self.area.window.draw_rgb_image(gc, 0,0, dx,dy, Gdk.RGB_DITHER_NONE, buffer(img_data_packed),3*pitch,0,0)
                         else: #grey
                             self.area.window.draw_gray_image(gc, 0,0, 
RefactoringTool: Files that need to be modified:
RefactoringTool: vcimgnetclient.py

2) pygi-convert

Please refer to General Porting Tips

1 2	longervision-GT72-6QE% ./pygi-convert.sh vcimgnetclient.py longervision-GT72-6QE%

3) Try Running

longervision-GT72-6QE% python vcimgnetclient.py 
vcimgnetclient.py:330: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "type" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  self.window =  Gtk.Window(Gtk.WindowType.TOPLEVEL)
vcimgnetclient.py:628: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  mfQuit =  Gtk.MenuItem("Quit")
vcimgnetclient.py:629: PyGIDeprecationWarning: Using non GObject arguments for connect_object() is deprecated, use: connect_data(signal, callback, data, connect_flags=GObject.ConnectFlags.SWAPPED)
  mfQuit.connect_object("activate", lambda a: Gtk.main_quit(), "mfQuit")
vcimgnetclient.py:637: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  bmf = Gtk.MenuItem("File")
vcimgnetclient.py:646: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  mhAbout =  Gtk.MenuItem("About")
vcimgnetclient.py:654: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  bmh = Gtk.MenuItem("Help")
Traceback (most recent call last):
  File "vcimgnetclient.py", line 812, in <module>
    Main()
  File "vcimgnetclient.py", line 719, in __init__
    Frontend.__init__(self, self.img_access)
  File "vcimgnetclient.py", line 301, in __init__
    self.winMain = self.WinMain(arg_img_access, self.dialog)
  File "vcimgnetclient.py", line 341, in __init__
    vbox.pack_start(self.menu_bar, False, False)
TypeError: Gtk.Box.pack_start() takes exactly 5 arguments (4 given)

4)

For ALL vbox.pack_start, add one parameter 0 at the end for each case. For instance:
vbox.pack_start(self.menu_bar, False, False)
to
vbox.pack_start(self.menu_bar, False, False, 0)

Oh my god … There are STILL SO MANY things to do, in order to make vcimgnetclient.py Python3 compatible. Therefore, I implement my own vcimgnetclient_qt.py based on PyQt5.

2.3.3 Longer Vision’s vcimgnetclient_qt.py

2.3.3.1 Server

22:42:22[root@VC-Z] /home/user
#whoami
root

22:42:24[root@VC-Z] /home/user
#vcimgnetsrv &
[1] 896

22:42:30[root@VC-Z] /home/user
#ps -e | grep vcimgnetsrv
  896 pts/0    00:00:00 vcimgnetsrv

22:42:40[root@VC-Z] /home/user
#vctp
Acquisiton&Copy Duration:          4s      39644us  for 100 Cycles ==  24.754656fps.

Acquisiton&Copy Duration:          4s      34709us  for 100 Cycles ==  24.784935fps.
......

2.3.3.2 Client

Sorry, I’m NOT going to show my code, but the performance can be demonstrated as follows:

Longer Vision Technology - Vision Component's Image Grabber - PyQt5

Kaggle Competition - APTOS 2019 Blindness Detection

Posted on 2019-08-26 | Edited on 1969-12-31 | In AI , Kaggle

Oh-my-god, kaggle competition about diabetis… Hmmm, I have something to do these days… It’s also a GOOD chance for me to learn some medical terms… In this blog, we’re going to try out this kaggle competition - APTOS 2019 Blindness Detection.

1. Pre-processing Diabetic Retinopathy Images

It’s NOT hard for us to locate APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, which seems to me a professional snippet of code preprocessing diabetic retinopathy images. Let’s just try it out directly.

1.1 Implementation

The implementation is cited directly from APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, with trivial modifications. You are welcome to download my snippet of code here.

1.2 Show Off Intermediately Processed Images

5 Categories	Diabetic Retinopathy Concepts

Original Images in Color	Original Images in Gray Scale

Ben Graham’s Preprocessing for Gray Scale Images	Ben Graham’s Preprocessing for Color Images

Circle Crop	Circle Crop High Resolution

Resized Train	Resized Train High Resolution

Some extentions about diabetic retinopathy are given in the end:

Heatmap	Inconsistency

Install Armbian Debian onto Longer Vision Technology H3 Board

Posted on 2019-08-25 | Edited on 2019-08-31 | In Single Board Computers , ARM

Today’s weather is perfect for this party: annual summer alumni party of Wuhan University.

Alumni	Me

Today, I’ve got another GOOD news: our ARM Linux board based on AllWinner H3 is demonstrated successful.

1. Hardware

2.1 Appearance

Yeah, this is how OUR OWN board looks like:

Longer Vision Technology AllWinner H3 Board

2.2 Specifications

From the appearance, it’s NOT hard to tell the specifications:

Device	Version	Units
USB	2.0	2
Wired	100Mbps	1
HDMI		1
Power Adaptor	5V	1

2. Flash Armbian Debian

2.1 Armbian for AllWinner H3

Armbian provides abundant info for AllWinner H3:

2.2 Armbian for Orange Pi PC

I’m so lucky today that I tried ONLY ONE TIME and have this Debian Buster with Armbian Linux for Orange Pi PC successfully installed and run on our board. You just need to download and extract Armbian_5.91_Orangepipc_Debian_buster_next_4.19.59.img out, and have it installed by dd.

2.3 Longer Vision Technology H3 Board Test

It’s NOT hard to find out your board’s IP address

either from within your router
or boot your board with a monitor and run the command ip -c address.

Now, you should be able to SSH into your H3 baord directly as follows:

SSH Into LVT H3 Board

After you run

sudo apt update and sudo apt upgrade
sudo armbian-config

you should be able to upgrade your Debian Buster with Armbian Linux from 4.19.59-sunxi to 4.19.62-sunxi.

longervision-GT72-6QE% ssh longervision@192.168.1.96
longervision@192.168.1.96's password: 
  ___  ____  _   ____   ____ 
 / _ \|  _ \(_) |  _ \ / ___|
| | | | |_) | | | |_) | |    
| |_| |  __/| | |  __/| |___ 
 \___/|_|   |_| |_|    \____|
                             
Welcome to Debian Buster with Armbian Linux 4.19.62-sunxi
System load:   0.16 0.12 0.08  	Up time:       6 min		
Memory usage:  18 % of 460MB  	IP:            192.168.1.96
CPU temp:      79°C           	
Usage of /:    2% of 58G    	

Last login: Sun Aug 25 20:13:29 2019 from 192.168.1.200

3. Issues

It’s a kind of late. Let me finish this up for now ASAP.

3.1 Hardware Issues

ONLY 1 USB 2.0 port is working properly.
It’s EASY to reach the critical temperature, as showing:

SSH Into LVT H3 Board

3.2

We really need a better low-cost chip to provide the support for:

USB 3.0
GigE

JeVois Smart Machine Vision Camera

Posted on 2019-08-23 | Edited on 2019-08-24 | In Machine Vision , ARMLinux

JeVois is an ARM Linux based camera module, delivered by JeVois Inc.. So far, this is the ONLY ARM Linux based camera module that I’m recommending, for its well-designed community as well as plenty of source code on Github.

1. Specifications

Clearly:

JeVois Hardware Page: JeVois is based on the Made In China ARM Linux chip Allwinners A33
JeVois Software Page: JeVois runs a full Linux operating system.

2. Getting Started with Jevois

The BEST place to start with JeVois is the following Youtube video:

I’m NOT going to repeat this step-by-step setup but move forward to the demonstration step directly.

3. Connect JeVois with Host

After having flashed JeVois’s TF card with the LATEST Linux image, we connect JeVois camera module into the laptop. This TF card will NOT mount onto the computer as a device automatically, but can be detected by lsusb as follows:

longervision-GT72-6QE% lsusb
......
Bus 001 Device 016: ID 1d6b:0102 Linux Foundation EEM Gadget
......

4. Simple Demonstration

4.1 Guvcview

Get Started	Image Control Default Settings

Video Control Settings 640x480	Video Control Settings 640x360

4.2 JeVois Inventor

4.2.1 JeVois Tutorial

Again, a Youtube video is suggested to follow for the demonstration:

4.2.2 My Test

You can either download JeVois Inventor directly from the Official Website, or like me, build JeVois Inventor source code from scratch, based on my own QT, etc. Anyway, after JeVois Inventor has been successfully installed, it’s the time for us to test it out.

JeVois Inventor Info	JeVois Inventor Parameters

JeVois Inventor Console	JeVois Inventor Camera

JeVois Inventor Config	JeVois Inventor Code C++

JeVois Inventor Code Python DNN	JeVois Inventor System

JeVois Inventor System Export SD Card	JeVois Inventor System Export SD Card Content

Being quite busy these days. Seriously, ARM board based on AllWinner H3 from my company Longer Vision is manufactured, and under test. Let’s have some fun.

In the end, better to listen to some music while blogging.

ESP32-Cam

Posted on 2019-08-22 | Edited on 2019-12-28 | In Machine Vision , MCU

0. Preface

My ESP32-Cam has finally arrived today. The brand of AI Thinker has been demonstrated to be reliable.

In fact, I strictly followed randomnerdtutorials’s blog ESP32-CAM Video Streaming and Face Recognition with Arduino IDE to demonstrate this low-cost video surveillance camera. So, in this blog, I ONLY enumerate some KEY POINTS that should be emphasized.

1. Connection

1.1 Schematic

ESP32-Cam can be connected to my laptop exactly the same as shown (cited directly from ESP32-CAM Video Streaming and Face Recognition with Arduino IDE).

ESP32-CAM-wiring-FTDI Schematic

1.2 Real Connection

The real connection looks as:

ESP32-CAM-wiring-FTDI Real Connection

1.3 lsusb

longervision-GT72-6QE% lsusb
......
Bus 001 Device 010: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC
......

2. Key Points on Flashing ESP32-Cam

2.1 Code Modification

Example code CameraWebServer should be trivially modified as:

// Select camera model
//#define CAMERA_MODEL_WROVER_KIT
//#define CAMERA_MODEL_ESP_EYE
//#define CAMERA_MODEL_M5STACK_PSRAM
//#define CAMERA_MODEL_M5STACK_WIDE
#define CAMERA_MODEL_AI_THINKER

2.2 Verify/Compile and Upload

2.2.1 Verify/Compile

1
2

Sketch uses 2241942 bytes (71%) of program storage space. Maximum is 3145728 bytes.
Global variables use 52696 bytes (16%) of dynamic memory, leaving 274984 bytes for local variables. Maximum is 327680 bytes.

2.2.2 Upload

esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP32D0WDQ5 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
MAC: c4:4f:33:13:32:7d
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 8192 bytes to 47...

Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 4102.7 kbit/s)...
Hash of data verified.
Compressed 16832 bytes to 10888...

Writing at 0x00001000... (100 %)
Wrote 16832 bytes (10888 compressed) at 0x00001000 in 1.0 seconds (effective 138.0 kbit/s)...
Hash of data verified.
Compressed 2242064 bytes to 1795627...

Writing at 0x00010000... (0 %)
Writing at 0x00014000... (1 %)
Writing at 0x00018000... (2 %)
Writing at 0x0001c000... (3 %)
Writing at 0x00020000... (4 %)
Writing at 0x00024000... (5 %)
Writing at 0x00028000... (6 %)
Writing at 0x0002c000... (7 %)
Writing at 0x00030000... (8 %)
Writing at 0x00034000... (9 %)
Writing at 0x00038000... (10 %)
Writing at 0x0003c000... (10 %)
Writing at 0x00040000... (11 %)
Writing at 0x00044000... (12 %)
Writing at 0x00048000... (13 %)
Writing at 0x0004c000... (14 %)
Writing at 0x00050000... (15 %)
Writing at 0x00054000... (16 %)
Writing at 0x00058000... (17 %)
Writing at 0x0005c000... (18 %)
Writing at 0x00060000... (19 %)
Writing at 0x00064000... (20 %)
Writing at 0x00068000... (20 %)
Writing at 0x0006c000... (21 %)
Writing at 0x00070000... (22 %)
Writing at 0x00074000... (23 %)
Writing at 0x00078000... (24 %)
Writing at 0x0007c000... (25 %)
Writing at 0x00080000... (26 %)
Writing at 0x00084000... (27 %)
Writing at 0x00088000... (28 %)
Writing at 0x0008c000... (29 %)
Writing at 0x00090000... (30 %)
Writing at 0x00094000... (30 %)
Writing at 0x00098000... (31 %)
Writing at 0x0009c000... (32 %)
Writing at 0x000a0000... (33 %)
Writing at 0x000a4000... (34 %)
Writing at 0x000a8000... (35 %)
Writing at 0x000ac000... (36 %)
Writing at 0x000b0000... (37 %)
Writing at 0x000b4000... (38 %)
Writing at 0x000b8000... (39 %)
Writing at 0x000bc000... (40 %)
Writing at 0x000c0000... (40 %)
Writing at 0x000c4000... (41 %)
Writing at 0x000c8000... (42 %)
Writing at 0x000cc000... (43 %)
Writing at 0x000d0000... (44 %)
Writing at 0x000d4000... (45 %)
Writing at 0x000d8000... (46 %)
Writing at 0x000dc000... (47 %)
Writing at 0x000e0000... (48 %)
Writing at 0x000e4000... (49 %)
Writing at 0x000e8000... (50 %)
Writing at 0x000ec000... (50 %)
Writing at 0x000f0000... (51 %)
Writing at 0x000f4000... (52 %)
Writing at 0x000f8000... (53 %)
Writing at 0x000fc000... (54 %)
Writing at 0x00100000... (55 %)
Writing at 0x00104000... (56 %)
Writing at 0x00108000... (57 %)
Writing at 0x0010c000... (58 %)
Writing at 0x00110000... (59 %)
Writing at 0x00114000... (60 %)
Writing at 0x00118000... (60 %)
Writing at 0x0011c000... (61 %)
Writing at 0x00120000... (62 %)
Writing at 0x00124000... (63 %)
Writing at 0x00128000... (64 %)
Writing at 0x0012c000... (65 %)
Writing at 0x00130000... (66 %)
Writing at 0x00134000... (67 %)
Writing at 0x00138000... (68 %)
Writing at 0x0013c000... (69 %)
Writing at 0x00140000... (70 %)
Writing at 0x00144000... (70 %)
Writing at 0x00148000... (71 %)
Writing at 0x0014c000... (72 %)
Writing at 0x00150000... (73 %)
Writing at 0x00154000... (74 %)
Writing at 0x00158000... (75 %)
Writing at 0x0015c000... (76 %)
Writing at 0x00160000... (77 %)
Writing at 0x00164000... (78 %)
Writing at 0x00168000... (79 %)
Writing at 0x0016c000... (80 %)
Writing at 0x00170000... (80 %)
Writing at 0x00174000... (81 %)
Writing at 0x00178000... (82 %)
Writing at 0x0017c000... (83 %)
Writing at 0x00180000... (84 %)
Writing at 0x00184000... (85 %)
Writing at 0x00188000... (86 %)
Writing at 0x0018c000... (87 %)
Writing at 0x00190000... (88 %)
Writing at 0x00194000... (89 %)
Writing at 0x00198000... (90 %)
Writing at 0x0019c000... (90 %)
Writing at 0x001a0000... (91 %)
Writing at 0x001a4000... (92 %)
Writing at 0x001a8000... (93 %)
Writing at 0x001ac000... (94 %)
Writing at 0x001b0000... (95 %)
Writing at 0x001b4000... (96 %)
Writing at 0x001b8000... (97 %)
Writing at 0x001bc000... (98 %)
Writing at 0x001c0000... (99 %)
Writing at 0x001c4000... (100 %)
Wrote 2242064 bytes (1795627 compressed) at 0x00010000 in 159.6 seconds (effective 112.4 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 134...

Writing at 0x00008000... (100 %)
Wrote 3072 bytes (134 compressed) at 0x00008000 in 0.0 seconds (effective 1534.2 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

2.3 Disconnect GPIO 0 from GND

Before testing the code, be sure to disconnect GPIO 0 from GND.

ESP32-CAM Disconnect

2.4 RESET ESP32-Cam Board

It’s VERY important that you open the Serial Monitor dialog to monitor the serial port. You will see the following outputs after resetting ESP32-Cam board.

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6400
entry 0x400806a8

[E][sccb.c:154] SCCB_Write(): SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:-1
.........
WiFi connected
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://192.168.1.94' to connect

2.5 Video Streaming

By visiting the allocated IP address within any web browser, you’ll see the following picture:

ESP32-CAM OV2460

And you’ll notice the streaming info from serial port:

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6400
entry 0x400806a8

[E][sccb.c:154] SCCB_Write(): SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:-1
.........
WiFi connected
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://192.168.1.94' to connect
MJPG: 5297B 63ms (15.9fps), AVG: 63ms (15.9fps), 0+0+0+0=0 0
MJPG: 5301B 40ms (25.0fps), AVG: 51ms (19.6fps), 0+0+0+0=0 0
MJPG: 5460B 39ms (25.6fps), AVG: 47ms (21.3fps), 0+0+0+0=0 0
MJPG: 5301B 40ms (25.0fps), AVG: 45ms (22.2fps), 0+0+0+0=0 0
MJPG: 5460B 43ms (23.3fps), AVG: 45ms (22.2fps), 0+0+0+0=0 0
MJPG: 5374B 41ms (24.4fps), AVG: 44ms (22.7fps), 0+0+0+0=0 0
MJPG: 5460B 37ms (27.0fps), AVG: 43ms (23.3fps), 0+0+0+0=0 0
MJPG: 5464B 43ms (23.3fps), AVG: 43ms (23.3fps), 0+0+0+0=0 0
MJPG: 5460B 37ms (27.0fps), AVG: 42ms (23.8fps), 0+0+0+0=0 0
MJPG: 5464B 46ms (21.7fps), AVG: 42ms (23.8fps), 0+0+0+0=0 0
MJPG: 5460B 30ms (33.3fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5464B 43ms (23.3fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5460B 39ms (25.6fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5464B 37ms (27.0fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5460B 37ms (27.0fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
......

3. Multiple Camera Streaming with Static IPs

I’m NOT going to talk about how to set up static IPs for IP cameras (well, yes, clearly, ESP32-Cam can be looked on as a low-cost video surveillance IP camera for sure). We ONLY demonstrate 4 ESP32-Cams are under streaming simultaneously with 4 statically allocated IP addresses.

We show the capturing effects FIRST:

4 ESP32-Cam Capturing Simultaneously

Then, we show how the cables are connected by using a power bank and an extra bread board. Here, FTDI is just used as a power adaptor to provide 5V voltage.

4 ESP32-Cams Connected to Power Bank

4 ESP32-Cams Cable Connection

Nobody

Longer Vision Technology Github Blog